StingyCD: Safely Avoiding Wasteful Updates in Coordinate Descent
ثبت نشده
چکیده
• Less time than an identical iteration of Algorithm 1 if q(t−1) ≤ τi and x i = 0 (the update is skipped) and rr is not updated. Specifically, StingyCD requires O(1) time, while CD requires O(NNZ (Ai)) time. • The same amount of time (up to an O(1) term) as a CD iteration if the update is not skipped and rr is not updated. In particular, both algorithms require the same number of O(NNZ (Ai)) operations. • More time than a CD iteration if rr is updated. In this case, StingyCD requires O(NNZ (A)) time.
منابع مشابه
StingyCD: Safely Avoiding Wasteful Updates in Coordinate Descent
Coordinate descent (CD) is a scalable and simple algorithm for solving many optimization problems in machine learning. Despite this fact, CD can also be very computationally wasteful. Due to sparsity in sparse regression problems, for example, often the majority of CD updates result in no progress toward the solution. To address this inefficiency, we propose a modified CD algorithm named “Sting...
متن کاملAvoiding communication in primal and dual block coordinate descent methods
Primal and dual block coordinate descent methods are iterative methods for solving regularized and unregularized optimization problems. Distributed-memory parallel implementations of these methods have become popular in analyzing large machine learning datasets. However, existing implementations communicate at every iteration which, on modern data center and supercomputing architectures, often ...
متن کاملFaster Coordinate Descent via Adaptive Importance Sampling
Coordinate descent methods employ random partial updates of decision variables in order to solve huge-scale convex optimization problems. In this work, we introduce new adaptive rules for the random selection of their updates. By adaptive, we mean that our selection rules are based on the dual residual or the primal-dual gap estimates and can change at each iteration. We theoretically character...
متن کاملPenalized Bregman Divergence Estimation via Coordinate Descent
Variable selection via penalized estimation is appealing for dimension reduction. For penalized linear regression, Efron, et al. (2004) introduced the LARS algorithm. Recently, the coordinate descent (CD) algorithm was developed by Friedman, et al. (2007) for penalized linear regression and penalized logistic regression and was shown to gain computational superiority. This paper explores...
متن کاملNon-homogeneous updates for the iterative coordinate descent algorithm
Statistical reconstruction methods show great promise for improving resolution, and reducing noise and artifacts in helical X-ray CT. In fact, statistical reconstruction seems to be particularly valuable in maintaining reconstructed image quality when the dosage is low and the noise is therefore high. However, high computational cost and long reconstruction times remain as a barrier to the use ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2017